{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Private Predictions with TFE Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Public Training without Differential Privacy\n",
    "\n",
    "Welcome to this tutorial! In the following notebooks you will learn how to provide private predictions. By private predictions, we mean that the data is constantly encrypted throughout the entire process. At no point is the user sharing raw data, only encrypted (that is, secret shared) data. In order to provide these private predictions, [TF Encrypted](https://github.com/tf-encrypted/tf-encrypted) combines cutting-edge cryptographic and machine learning techniques. But you don't have to worry about this and can focus on your machine learning application.\n",
    "\n",
    "You can start serving private predictions with only three steps:\n",
    "- **Step 1**: train your model with normal Keras.\n",
    "- **Step 2**: secure and serve your machine learning model (server).\n",
    "- **Step 3**: query the secured model to receive private predictions (client). \n",
    "\n",
    "Alright, let's go through these three steps so you can deploy impactful machine learning services without sacrificing user privacy or model security."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train your model with Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use privacy-preserving machine learning techniques for your projects you should not have to learn a new machine learning framework. If you have basic [Keras](https://keras.io/) knowledge, you can start using these techniques with Syft Keras. If you have never used Keras before, you can learn a bit more about it through the [Keras documentation](https://keras.io). \n",
    "\n",
    "Before serving private predictions, the first step is to train your model with normal Keras. As an example, we will train a model to classify handwritten digits. To train this model we will use the canonical [MNIST dataset](http://yann.lecun.com/exdb/mnist/).\n",
    "\n",
    "We borrow [this example](https://github.com/keras-team/keras/blob/master/examples/mnist_cnn.py) from the reference Keras repository.  To train your classification model, you just run the cell below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Public Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-11-01 10:02:42.046579: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2022-11-01 10:02:42.050310: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n",
      "2022-11-01 10:02:42.050322: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import tensorflow as tf\n",
    "\n",
    "from tensorflow.keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n",
      "Epoch 1/2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-11-01 10:02:43.744611: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n",
      "2022-11-01 10:02:43.744637: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "2022-11-01 10:02:43.744651: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (mpc-empty011122134151.ea134): /proc/driver/nvidia/version does not exist\n",
      "2022-11-01 10:02:43.744933: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "469/469 [==============================] - 4s 7ms/step - loss: 0.5165 - accuracy: 0.8467 - val_loss: 0.1849 - val_accuracy: 0.9455\n",
      "Epoch 2/2\n",
      "469/469 [==============================] - 3s 7ms/step - loss: 0.1558 - accuracy: 0.9534 - val_loss: 0.1110 - val_accuracy: 0.9657\n",
      "Test loss: 0.11101692169904709\n",
      "Test accuracy: 0.9656999707221985\n"
     ]
    }
   ],
   "source": [
    "batch_size = 128\n",
    "num_classes = 10\n",
    "epochs = 2\n",
    "\n",
    "# input image dimensions\n",
    "img_rows, img_cols = 28, 28\n",
    "\n",
    "# the data, split between train and test sets\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
    "x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
    "input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "print('x_train shape:', x_train.shape)\n",
    "print(x_train.shape[0], 'train samples')\n",
    "print(x_test.shape[0], 'test samples')\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "y_train = tf.keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = tf.keras.utils.to_categorical(y_test, num_classes)\n",
    "\n",
    "model = tf.keras.Sequential([\n",
    "          tf.keras.layers.Conv2D(16, 8,\n",
    "                                 strides=2,\n",
    "                                 padding='same',\n",
    "                                 activation='relu',\n",
    "                                 input_shape=(28, 28, 1)),\n",
    "          tf.keras.layers.AveragePooling2D(2, 1),\n",
    "          tf.keras.layers.Conv2D(32, 4,\n",
    "                                 strides=2,\n",
    "                                 padding='valid',\n",
    "                                 activation='relu'),\n",
    "          tf.keras.layers.AveragePooling2D(2, 1),\n",
    "          tf.keras.layers.Flatten(),\n",
    "          tf.keras.layers.Dense(32, activation='relu'),\n",
    "          tf.keras.layers.Dense(10, activation='softmax')\n",
    "  ])\n",
    "\n",
    "model.compile(loss=tf.keras.losses.categorical_crossentropy,\n",
    "              optimizer=tf.keras.optimizers.Adam(),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit(x_train, y_train,\n",
    "          batch_size=batch_size,\n",
    "          epochs=epochs,\n",
    "          verbose=1,\n",
    "          validation_data=(x_test, y_test))\n",
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save your model's weights for future private prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Your model is trained. Let's save the model weights with `model.save()`. In the next notebook, we will load these weights in Syft Keras to start serving private predictions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('short-dnn.h5')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('tfe2')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "2b3cf1a821f25a2bafd97d4b4f3a77d0e1f42d8d78c0e512d3a72b703c70416e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
